Update `collect` to take input with specified format by davidallendj · Pull Request #130 · OpenCHAMI/magellan

davidallendj · 2026-04-06T22:11:34Z

Pull Request Template

Thank you for your contribution! Please ensure the following before submitting:

Checklist

My code follows the style guidelines of this project
I have added/updated comments where needed
I have added tests that prove my fix is effective or my feature works
I have run make test (or equivalent) locally and all tests pass
DCO Sign-off: All commits are signed off (git commit -s) with my real name and email
[] REUSE Compliance:
- Each new/modified source file has SPDX copyright and license headers
- Any non-commentable files include a <filename>.license sidecar
- All referenced licenses are present in the LICENSES/ directory

Description

This PR changes the collect command to take JSON or YAML as input from the scan command. This allows for the entire discovery process to be completed as a single command without hitting the file system.

Examples:

# scan -> collect
magellan scan --subnet 172.18.0.0/24 --port 5000 -l info -i -F json | magellan collect -f json --show-output -i

# scan -> collect -> send
magellan scan --subnet 172.18.0.0/24 --port 5000 -l info -i -F json | magellan collect -f json --show-output -i | magellan send https://smd.example.com

# collect -> send
magellan collect pdu x3000m0 x3000m1 -u admin -p password | magellan send https://smd.example.com

Fixes #128

Type of Change

Bug fix
New feature
Breaking change
Documentation update

For more info, see Contributing Guidelines.

Signed-off-by: David Allen <davidallendj@gmail.com>

synackd

Testing with real BMCs.

Scanning:

$ ./magellan scan --subnet 172.16.0.0/24 -l info -i -F json --insecure
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:45:41-06:00","message":"failed to register completion function"}
[
  {
    "host": "https://172.16.0.105",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.948173759-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.104",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947999177-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.102",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947644981-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.101",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947476248-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.103",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947691662-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.199",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.948031377-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.198",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947765703-06:00",
    "service_type": "Redfish"
  }
]

Scanning piping to collect:

$ ./magellan scan --subnet 172.16.0.0/24 -l info -i -F json --insecure | ./magellan collect -f json --show-output -i
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:46:07-06:00","message":"failed to register completion function"}
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:46:07-06:00","message":"failed to register completion function"}
{"level":"error","error":"no file found","time":"2026-05-14T13:46:07-06:00","caller":"/opt/shared/magellan/cmd/collect.go:61","message":"failed to get scanned results from cache"}
{"level":"error","error":"no assets found","time":"2026-05-14T13:46:07-06:00","caller":"/opt/shared/magellan/cmd/collect.go:178","message":"failed to collect data"}
null

synackd · 2026-04-07T14:58:33Z

@@ -9,16 +9,20 @@ import (
 	"github.com/OpenCHAMI/magellan/internal/cache/sqlite"
 	"github.com/OpenCHAMI/magellan/internal/format"
 	magellan "github.com/OpenCHAMI/magellan/pkg"


If we haven't added linting to Magellan yet, we definitely should. Internal imports should go after other package imports. I suspect this needs to be done in other files as well, so it can be addressed in a subsequent PR.

davidallendj · 2026-06-01T20:01:40Z

Testing with real BMCs.

Scanning:

$ ./magellan scan --subnet 172.16.0.0/24 -l info -i -F json --insecure
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:45:41-06:00","message":"failed to register completion function"}
[
  {
    "host": "https://172.16.0.105",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.948173759-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.104",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947999177-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.102",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947644981-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.101",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947476248-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.103",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947691662-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.199",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.948031377-06:00",
    "service_type": "Redfish"
  },
  {
    "host": "https://172.16.0.198",
    "port": 443,
    "protocol": "tcp",
    "state": true,
    "timestamp": "2026-05-14T13:45:41.947765703-06:00",
    "service_type": "Redfish"
  }
]

Scanning piping to collect:

$ ./magellan scan --subnet 172.16.0.0/24 -l info -i -F json --insecure | ./magellan collect -f json --show-output -i
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:46:07-06:00","message":"failed to register completion function"}
{"level":"warn","error":"RegisterFlagCompletionFunc: flag 'format' does not exist","time":"2026-05-14T13:46:07-06:00","message":"failed to register completion function"}
{"level":"error","error":"no file found","time":"2026-05-14T13:46:07-06:00","caller":"/opt/shared/magellan/cmd/collect.go:61","message":"failed to get scanned results from cache"}
{"level":"error","error":"no assets found","time":"2026-05-14T13:46:07-06:00","caller":"/opt/shared/magellan/cmd/collect.go:178","message":"failed to collect data"}
null

I think this may be happening because the --cache-path flag is set to a non-empty string by default.

This is where the check happens:
https://github.com/OpenCHAMI/magellan/blob/allend/collect-json-input/cmd/collect.go#L58

And here is where the flag default is set:
https://github.com/OpenCHAMI/magellan/blob/allend/collect-json-input/cmd/root.go#L93

I can run it by setting --cache "" but I'm running to another problem that I'm working to address.

./magellan scan --subnet 172.16.0.0/24 -l info -i -F json | ./magellan collect -f json --show-output -i --cache "" -l debug

Signed-off-by: David Allen <davidallendj@gmail.com>

davidallendj · 2026-06-01T21:01:34Z

@synackd It should be working now. I had to add some missing logic back to cmd/collect.go and update some of the CLI flags. I don't think I like having to explicitly set --cache "" for this to work though...

This is the command I'm using to test currently:

./magellan scan --subnet 172.16.0.0/24 -l info -i -F json | ./magellan collect -f json --show-output -i --cache "" -l debug

synackd · 2026-06-01T21:57:30Z

That command line worked for me. What if we added a check to see that stdin was open and, if so, used that instead of a cache unless explicitly told to do so?

davidallendj added 10 commits April 6, 2026 16:23

chore: updated go deps

6d92810

Signed-off-by: David Allen <davidallendj@gmail.com>

refactor: minor changes to scan plus added comments

e245f50

Signed-off-by: David Allen <davidallendj@gmail.com>

refactor: reworked cmd/collect.go input and removed token logic

b2e5def

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: sqlite driver issue

ad2970d

Signed-off-by: David Allen <davidallendj@gmail.com>

refactor: removed unnecessary code

a251de4

Signed-off-by: David Allen <davidallendj@gmail.com>

tests: updated cmd and tests to work with changes

59f7367

Signed-off-by: David Allen <davidallendj@gmail.com>

refactor: make crawl output flag consistent with collect

55a7211

Signed-off-by: David Allen <davidallendj@gmail.com>

docs: updated man pages

82f59f0

Signed-off-by: David Allen <davidallendj@gmail.com>

docs: updated man pages

0d16ae1

Signed-off-by: David Allen <davidallendj@gmail.com>

docs: updated examples in cmd

92bdb3b

Signed-off-by: David Allen <davidallendj@gmail.com>

davidallendj force-pushed the allend/collect-json-input branch from e59ee6e to 92bdb3b Compare April 6, 2026 22:24

docs: updated README about workflows

419048b

Signed-off-by: David Allen <davidallendj@gmail.com>

davidallendj requested review from alexlovelltroy and synackd April 7, 2026 14:42

synackd reviewed May 14, 2026

View reviewed changes

davidallendj added 4 commits June 1, 2026 14:24

refactor: minor changes to CLI

4020578

Signed-off-by: David Allen <davidallendj@gmail.com>

refactor: minor changes to pkg and update debug message

e311697

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: removed files from other branch

b284ab8

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: issue with registering format completion funcs

7b1e8d4

Signed-off-by: David Allen <davidallendj@gmail.com>

davidallendj force-pushed the allend/collect-json-input branch from cbe75b6 to 7b1e8d4 Compare June 1, 2026 20:32

davidallendj added 4 commits June 1, 2026 14:34

fix: power.format cobra/viper binding

2d3bb6a

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: added missing append for incoming assets in cmd/collect.go

7c08a48

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: added missing code to append assets from flag

4ecc252

Signed-off-by: David Allen <davidallendj@gmail.com>

fix: changed asset to pointer for format.UnmarshalData

255ba34

Signed-off-by: David Allen <davidallendj@gmail.com>

synackd changed the title ~~Update collect take input with specified format~~ Update collect take input with specified format Jun 1, 2026

synackd changed the title ~~Update collect take input with specified format~~ Update collect to take input with specified format Jun 1, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update `collect` to take input with specified format#130

Update `collect` to take input with specified format#130
davidallendj wants to merge 19 commits into
mainfrom
allend/collect-json-input

davidallendj commented Apr 6, 2026 •

edited by synackd

Loading

Uh oh!

synackd left a comment

Uh oh!

synackd Apr 7, 2026

Uh oh!

davidallendj commented Jun 1, 2026

Uh oh!

davidallendj commented Jun 1, 2026

Uh oh!

synackd commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

davidallendj commented Apr 6, 2026 • edited by synackd Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Template

Checklist

Description

Type of Change

Uh oh!

synackd left a comment

Choose a reason for hiding this comment

Uh oh!

synackd Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

davidallendj commented Jun 1, 2026

Uh oh!

davidallendj commented Jun 1, 2026

Uh oh!

synackd commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

davidallendj commented Apr 6, 2026 •

edited by synackd

Loading